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A multipoint multimedia server (130) includes a number of different compression codecs (134), a multipoint switch (132), separate 
audio (136) and video (138) processors, and at least one controller (140). Data streams of different compression standards enter the server 
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INTERCONNECTING MULTIMEDIA DATA STREAMS HAVING DIFFERENT 
COMPRESSED FORMATS 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates broadly to multimedia telecommunications. More 
particularly, the present invention relates to methods, apparatus, and systems for the handling of 
compressed multimedia communication data so that multimedia equipment utilizing different 
data compression formats can be interconnected with each other. 

2. State of the Art 

With the advent of the optical network the telecommunications bandwidth available to 
business and individuals has increased dramatically. One concept utilizing the increased 
bandwidth is video communication (more commonly now called "multimedia" communications) 
which permits video, audio, and in some cases other data to be transported from one party to 
another or others. Multimedia communications can be utilized for a number of applications, and 
in different configurations. One configuration of recent interest has been multimedia 
conferencing, where several parties can communicate in a conference style. 

In multimedia conferencing, the video data is handled such that each party can see at 
least one of the other parties, while the audio data is handled such that each party can hear one, 
several, or all of the other parties. In fact, various telecommunications standards are presently 
being adopted by the ITU-T and ISO which govern the protocols of multimedia conferencing 
(see, e.g., H.320, ITU-T. 120). As part of the multimedia conferencing, various standards 
have been adopted for the compression of the video and audio data. For example, among 
compression standards applied to video compression are the JPEG (Joint Photographic Experts 
Group) standards promulgated by the joint ISO/CCTTT technical committee ISO/EC 
JTC1/SC2/WG10, and the MPEG (Motion Picture Experts Group) standards promulgated by 
ISO under ISO/IEC 1 1 172(MPEG-1) and ISO/IEC 13818(MPEG-2). Another compression 
standard known as H.261 provides a very high compression ratio. Among the audio 
compression standards are the G722, G728 and MPEG audio compression. In addition, other 
compression techniques such as ADPCM (adaptive differential pulse code modulation) are 
known. 
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In the multimedia conferencing systems of the art (as represented by prior art Figure 1), 
the audio, video, and other data streams generated by a user's system 12a are multiplexed 
together directly in the encoder section of a multimedia 

encoder/decoder (codec) 14 located at the source/terminal 16, and transported together as an 
indivisible stream through the transport network 20 (now proposed in ATM format) to a similar 
"peer" codec 24 at a remote location. The peer codec is either at the remote user site for a point- 
to-point conference, and/or at a multimedia bridge 26 for a multipoint conference. The 
multimedia bridge 26, which typically includes a codec/switch 24 and a controller 28, provides 
conference control (e.g., it determines the signal to be sent to each participant), audio mixing 
(bridging) and multicasting, audio level detection for conference control, video switching, video 
mixing (e.g., a quad split, or "continuous presence device" which combines multiple images for 
display together) when available and/or desirable, and video multicasting. The audio and video 
data exiting the multimedia bridge is multiplexed, and continues through the transport network 
20 to the desired multimedia source terminals 12b, 12c. 

While the multimedia conference systems of the art are generally suitable for multimedia 
conferencing purposes, presently each terminal 12 connected to the conference must use a 
compatible codec 1 4. For example, a terminal using a Motion JPEG codec cannot communicate 
with a terminal using an MPEG or 

H.320 codec. Similarly, a terminal using an MPEG codec cannot communicate with a terminal 
using a Motion JPEG or H.320 codec, etc. 

There have been recent proposals to incorporate "transcoders" within the fabric of an 
ATM network or at a user site. The proposed transcoders would have the ability to input a data 
stream, determine the compression standard being used, and, with the use of a transcoding 
algorithm, output a data stream which uses a different compression standard. To date, 
however, there has been little success in providing an acceptable transcoder. The algorithms 
necessary to convert one compression standard to another are difficult to design. The 
algorithms provided thus far have exhibited severe degradation of the output data stream. In 
addition, switching and mixing of multimedia data is rendered more difficult when using a 
transcoder. 
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SUMMARY OF THE INVENTION 

It is therefore an object of the present invention to provide methods, apparatus, and 
systems for interconnecting compressed multimedia communication data streams having 
different compression formats. 

It is another object of the invention to provide methods, apparatus, and systems for 
converting a data stream compressed with one standard to a data stream compressed with a 
different standard. 

It is a further object of the invention to provide methods, apparatus, and systems which 
allow mixing and switching data from data streams compressed with different standards. 

It is an additional object of the invention to provide methods, apparatus, and systems for 
converting a data stream compressed with one standard to a data stream compressed with a 
different standard wherein the output data stream signal quality is not substantially degraded. 

In accord with these objects which will be discussed in detail below, the methods, 
apparatus, and systems according to the invention include providing a multipoint multimedia 
server which includes a plurality of different compression-scheme related codecs, a multipoint 
switch, separate audio and video processors, and at least one controller. Data streams of 
different compression standards from different users enter the server wherein the data streams 
are directed to the appropriate codec and are decoded to either baseband analog form, or more 
preferably to decompressed digital form. The decompressed digital or baseband analog signals 
are mixed and switched by the controller(s) and the multipoint switch according to user 
commands and are then routed back to appropriate codecs where the mixed and switched signals 
are recompressed to the appropriate standard for each user before exiting the server. The server 
is typically located at a remote node which is part of the communications network. The server, 
according to the invention preferably includes codecs for MPEG, Motion JPEG, and H.320 
compression standards and may include other codecs based on proprietary schemes, as desired. 
While the presendy preferred embodiment of the invention converts all of the data streams to 
analog audio and video prior to mixing and switching, in accord with an alternate embodiment, 
an appropriate digital multipoint switch can enable the data streams to be mixed and switched in 
decompressed (baseband) digital form. The multimedia terminals served by the invention may 
utilize NTSC, PAL, or SECAM video standards and the data streams may include other data as 
well as audio and video. 
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According to a preferred embodiment of the invention, the server is coupled to an ATM 
network. However, the server according to the invention may be used with other networks, 
either LAN or WAN. 

Additional objects and advantages of the invention will become apparent to those skilled 
in the art upon reference to the detailed description taken in conjunction with the provided 
figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a high level diagram of a prior art multimedia conferencing system having a 
plurality of same-compression-type multimedia conferencing sites coupled by an ATM 
telecommunications network; 

Figure 2 is a high level diagram of a multipoint multimedia server according to the 
invention coupled to a node of an ATM network serving a plurality of multimedia conferencing 
sites each utilizing a different data compression standard; 

Figure 2a is a block diagram of the functionality of a portion of the server of Figure 2; 

Figures 2b and 2c are flow diagrams of, respectively, the audio processor and video 
processor of the server of Figure 2a; 

Figure 3a is a block diagram of a Motion JPEG codec used in the server according to the 
invention; 

Figure 3b is a block diagram of an MPEG codec used in the server according to the 
invention; 

Figure 3c is a block diagram of an H.320 codec used in the server according to the 
invention; 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 



Turning now to Figure 2, a first system 100 according to the invention services a 
plurality of users 1 12a, 1 12b, 1 12c, each of which are each provided with codecs 1 14a, 1 14b, 
114c, and multimedia source/terminals 116a, 116b, 116c. The codecs 114c- 114c, which are 
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described in greater detail with reference to Figures 3a-3c act as an interface between the 
network 120, and the source/terminals 1 16. The source/terminals typically include cameras 
117, microphones 118, and computer terminals 119. The computer terminals 119,intum, 
typically include a video monitor 1 19a, speakers 1 19b, keyboards 1 19c, etc. as is known in the 
art. As shown in Figure 2, user 1 12a is provided with a Motion JPEG codec 1 14a, user 1 12b 
is provided with an MPEG codec 1 14b, and user 1 12c is provided with an H.320 codec 1 14c. 
It will be appreciated that the network 120 is preferably an ATM network and, as such, the 
H.320 codec 1 14c is coupled to the network 120 via an ISDN/ATM gateway 115. It will 
further be appreciated that each of the users 1 12a- 1 12c, is coupled to the network 120 at a node 
122a, 122b, 122c. The system 100 also includes a multipoint multimedia server 130 which 
may be located anywhere in the ATM network 120. 

The server 130 according to the invention, includes an ATM switch 132, a plurality of 
codecs 134a, 134b, 134c... 134n, an audio processor 136, a video processor 138, and a 
controller 140. As shown, the codec 134a is a Motion JPEG codec, the codec 134b is an 
MPEG-2 codec, and the codec 134c is an H.320 codec. The codecs are described in detail 
below with reference to Figures 3a-3c. Generally, however, each codec receives an appropriate 
data stream via the switch 132 and passes a decoded signal (either baseband analog or 
decompressed digital) to the audio processor 136 and the video processor 138. 

As seen in Fig. 2a, where the codecs provide baseband analog signals to the audio 
processor 136, the audio analog signals are converted by an A/D converter 136a into digital 
samples which are fed via a dedicated bus 136b to a DSP 136c. The digital samples are then 
processed by the DSP 136c (as discussed in more detail below with reference to Fig. 2b). For 
example, the audio information (samples) may be mixed under control of the controller 140 with 
the audio information of other users of the multimedia conference. Typically, the audio mixing 
involves summing, or weighting and summing the audio of all of the audio signals of the users 
in the conference, and then subtracting the source audio signal of the destination. After 
processing by the DSP 136c, the mixed audio samples are sent via the bus 136b for conversion 
to a baseband analog signal by a D/A converter 136d. The so-obtained mixed audio information 
may then be coded by the appropriate codec and forwarded to its destination via the ATM switch 
132. 

Where the codecs provide baseband analog video signals to the video processor 138, the 
video processor 138 may, under control of system controller 140, simply route (via switch 
138a) the video information to the codecs, or may multiplex video information using the split 
processor 138b for split or multi screen transmission to its destination(s). Details of the video 
processing are discussed in more detail below with reference to Fig. 2c. 
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At each destination, the audio and video information being received is provided to the 
user's codec 1 14 in the proper compression format for processing, and then provided to the 
video monitor 1 19a and to the speaker 1 19b. 

As seen in Fig. 2a, the server 130 also includes a data processing subsystem 137 which 
preferably utilizes an Ethernet interface between the codecs 134 and its T.12x stack 137a-137e. 
The data processing subsystem 137 can receive conference creation, scheduling, and 
management information which is passed to the system controller 140. As indicated, the system 
controller 140 provides various functions including node control and security 140a, operations, 
administration and management functions 140b, audio and video control 140c, and conference 
scheduling 140d. The audio and video control function 140c interfaces with the audio and 
video processors 136 and 138 as previously described. 

Turning now to Fig. 2b, a flow chart of the audio processor function is seen. Thus, at 
150, the audio processor initializes itself by setting up buffers, pointers, and assigning default 
register values. Then, at 152, the audio processor awaits commands from the system controller 
140. Upon receiving at 154 a command, the audio processor DSP validates the command at 
156 and processes the command at 158 and sets required flags for the interrupt service routine. 

The interrupt service routine of the audio processor DSP 136c (entered at n/f intervals 
where n is the number of buffered audio samples per port and f is the sampling frequency) starts 
at 160 with the DSP obtaining from the A/D converters 136a the audio samples available at all 
ports and storing them into a matrix S. At 162, for each port P which is inactive or muted, all 
samples for that port are set to zero in the matrix; i.e., S[P] = 0. For all of the ports relating to a 
conference (i.e., a group), the samples are added together at 164; i.e., T(g) = £S[P] for the 
ports of that group. At 166, the power P[p] of each port is computed. Then, at 168, for each 
conference, the loudest port L[p] is found by comparing the powers P[p] for that group. At 
170, for each port which is subject to an automatic gain control, the gain is adjusted according 
to the effective and targeted power values as described in co-owned, copending U.S. Patent 
Application SN 08/888,571 to Tabet et al., fded July 7, 1997, which is hereby incorporated by 
reference herein in its entirety. At 172, for each port, the sample vector for that port is 
subtracted from the sum of the group vectors; i.e., S[P] = T[g] - S[P], to provide a mixed audio 
sample for output. The mixed audio samples are then sent out at 174 for digital to analog 
conversion. 

Turning to Fig. 2c, a flow chart of the video processing function is provided. At 180, 
the video system initializes itself. At 182, the video system awaits a command from the 
controller 140. Upon receiving at 184 a command, the video switch is formatted at 186 such 
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that any input (whether from the codecs or the split processor 138b) can be switched to any 
output (including the codecs and the split processor). 

A high level block diagram of the Motion JPEG codec 1 14a (134a) is seen in Figure 3a. 
The Motion JPEG codec generally includes audio circuitry 200, video circuitry 220, modem 
type data circuitry 240, and a main processor or host 250 with an associated bus 252 and 
associated circuitry such as a boot PROM 254, flash memory 256, SRAM 258, a local manager 
interface 260, a timer 262, a watchdog circuit 264, and a management port UART 266. The 
boot PROM 254 stores the code which boots the main processor 250. The flash memory 256 is 
typically utilized to hold the main code and static configuration information. The RAM 258 is 
typically a dynamic RAM for running the code and temporarily storing data. The timer 262 
provides clock signals for the operating system, while the watchdog 264 performs reset 
functions. The management port UART 266 is provided for access by a system manager (not 
shown) to the codec, while the local management interface 260 provides the codec with the 
capability of interfacing with a local manager such as the controller 140 of Figure 2. 

The audio circuitry 200 includes an analog to digital converter 202, an audio codec 
interface 204, an audio source time manager 206, an audio packet buffer SRAM 210, a packet 
buffer and DMA controller 212, an audio channel output interface 214, an audio channel input 
interface 216, an audio presentation time manager 217, and a digital to analog converter 218. 
The video circuitry 220 includes a composite NTSC/PAL video decoder 222, a JPEG 
compressor 224, a video source time manager 226, an outgoing packet buffer SRAM 227, an 
outgoing video packet buffer and DMA controller 228, a video channel output interface 229, 
and a video channel input interface 231, an incoming packet buffer and DMA controller 232, an 
incoming packet buffer SRAM 233, a JPEG decompressor 234, a composite NTSC/PAL video 
encoder 236, a video presentation time manager 238, and a video sync generator 239. The data 
circuitry 240, includes a data channel port UART 242, a data packet buffer and DMA controller 
244 with an associated data packet buffer RAM 245, a data channel output interface 246, and a 
data channel input interface 248. 

Generally, outgoing audio information received from a microphone(s) or other audio 
source is applied to the analog to digital converter 202 which simultaneously provides the digital 
audio data to the audio codec interface 204 and, in accord with a preferred aspect of the 
invention, provides a reference clock to the audio source time manager 206. The audio codec 
interface 204 converts the format of the data received from the A/D converter so that the data 
may be properly provided to the packet buffer SRAM 210 under control of the packet buffer and 
DMA (direct memory access) controller. In addition, in accord with the preferred embodiment, 
the main processor 250 provides a PES (Program Elementary Stream) header to the SRAM 210 
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to effectively generate PES formatted packet. The packet buffer and DMA controller 212 
controls the movement of the packetized audio data from the SRAM 210 to the channel output 
interface 214 as required. The channel output interface 214, in turn, places the data in a desired 
format (e.g., a Transport Stream (TS) format, or an ATM format) by inserting a system time 
indicator (provided by the audio source time manager) into the signal, and provides the desired 
overhead bits or bytes (including OAM where appropriate). The channel output interface 214 
implements a serial channel physical interface by receiving the parallel stream of data from the 
buffer controller, and converting the parallel stream into a serial stream with an accompanying 
clock, etc., which is multiplexed with video data by the multiplexer/demultiplexer 270. 

Incoming audio information is received by the audio channel input interface 216 from 
the multiplexer/demultiplexer 270. The audio channel input interface 216 frames on the 
incoming (TS) cell, checks the headers for errors, passes the payload in byte wide format 
(parallel format) to the packet buffer and DMA controller 212, and passes a time reference 
marker (Program Clock Reference value -PCR) to the audio presentation time manager 217. 
The DMA controller 212 places the payload in desired locations in the SRAM 210. When the 
data is to be presented to the receiver as audio information, the DMA controller takes the data 
out of the SRAM 210 and provides it to the audio codec interface 204, which reformats the data 
into a serial stream for digital to analog conversion by the D/A converter 218. The presentation 
time manager 217 is provided to recover a local clock. 

The video circuitry 220 processes and outputs the video signals. Outgoing video 
information is received by the video circuitry 220 as a composite analog input. The composite 
input is decoded by the composite decoder 222 which provides digital luminance and color 
difference signals to the video compressor 224, and horizontal and vertical synch and a field 
indicator to the video source time manager 226. The Motion JPEG video compressor 224 
compresses the data, and generates a JPEG frame with start of image and end of image markers. 
The video compressor 224 puts the framed compressed data in parallel format, so that the buffer 
controller 228 can place the compressed data into the packet buffer SRAM 227. Preferably, the 
host (main processor 250) provides PES headers via the buffer controller 228 to desired 
locations in the SRAM 227 to effectively convert the JPEG frame into a PES packet The 
packet buffer and DMA controller 228 provides the channel output interface 229 with the PES 
packet data at a constant rate. If sufficient data is not available in the packet buffer SRAM 227, 
the channel output interface 229 generates an "idle cell". Regardless, the channel output 
interface 229, places the data in a desired format (e.g., TS or ATM format) by inserting a 
system time indicator (provided by the video source time manager 226) into the signal, and 
provides the desired overhead bits or bytes (including OAM where appropriate). The channel 
output interface 229 implements a serial channel physical interface by receiving the parallel 
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stream of data from the buffer controller 228, and converting the parallel stream into a serial 
stream with an accompanying clock, etc. The outgoing stream of video data is multiplexed with 
time-related audio data at the multiplexer/demultiplexer 270. 

In the receiving direction, video data which is demultiplexed by the 
multiplexer/demultiplexer 270 is obtained at the video channel input interface 231 which frames 
on the incoming (TS) cell, checks the headers for errors, passes the payload in byte wide format 
(parallel format) to the packet buffer and DMA controller 232, and passes a time reference 
marker to the video presentation time manager 238. The DMA controller 232 places the payload 
in desired locations in the SRAM 233. When the JPEG decompressor 234 indicates that the 
next video field is required for display, the DMA controller 232 provides the buffered 
compressed video from the head of the buffer to the JPEG decompressor 234. The JPEG 
decompressor 234, in turn, decompresses the data, and provides digital luminance and color 
difference signals to the composite video encoder 236. 

The composite video encoder 236 operates based on the video timing signals (horizontal 
line timing or H, vertical field timing or V, and the field indicator) generated by the sync 
generator, based on the sample clock recovered by the presentation time manager from the 
channel PCR. The composite video encoder 236 in turn indicates to the JPEG decompressor 
when it requires video data for the next video field, and which field is required. Based on these 
timing signals and the decompressed video data, the composite video encoder generates the 
analog video output for the video monitor 1 19a or for the video processor 138. 

Turning now to Figure 3b, an MPEG-2 codec 1 14b, 134b generally includes audio 
circuitry 300, video circuitry 320, controller circuitry 340, management port circuitry 350, 
multiplexer/demultiplexer circuitry 360, and channel interface circuitry 370. The audio circuitry 
300 includes an analog to digital converter 302, a digital to analog converter 304, an 
MPEG-2 audio processor (compressor/decompressor) 306, and an audio buffer controller 308. 
The video circuitry 320 includes a composite NTSC/PAL video decoder 322, an MPEG-2 
compressor 324, an outgoing video buffer controller 326, an incoming video buffer controller 
328, an MPEG decompressor 330, and a composite NTSC/PAL video encoder 332. The 
controller circuitry 340 is coupled to the buffer controllers 308, 326, 328 as well as to the 
management port circuitry 350. The multiplexer/demultiplexer circuitry 360 is coupled to the 
buffer controllers 308, 326, 328 as well as to the channel interface circuitry 370. 

Generally, incoming MPEG-2 coded audio/video data is received via the channel 
interface 370 and split into separate audio and video signal streams by the 
multiplexer/demultiplexer circuitry 360. The inomiing audio data is passed to the audio buffer 
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controller 308 which provides the audio data to the MPEG-2 audio processor 306. The MPEG- 
2 audio processor 306 decompresses the audio data and provides baseband (decompressed) 
digital data which is converted by the digital to analog converter 304 into an analog audio 
output The incoming video data is likewise passed to the video decoder buffer controller 328 
which provides the video data to the MPEG-2 video decoder 330. The MPEG-2 video decoder 
330 decompresses the video data and provides decompressed digital data which is converted by 
the video encoder 332 into a composite video output. 

On the outgoing side, outgoing audio information received from a microphone(s) or 
other audio source (e.g. an audio mixer at the server) is applied to the analog to digital converter 
302 which provides a digital output to the MPEG-2 audio processor 306. The MPEG-2 audio 
processor 306 in turn provides compressed digital audio data to the audio buffer controller 308. 
Likewise, the outgoing video data received from a camera or other video source (e.g. a video 
switch or mixer) in the form of composite video is applied to the Video decoder 312 which 
provides a digital signal to the MPEG-2 video encoder 324. The compressed digital data output 
of the MPEG-2 encoder 324 is passed through the buffer controller 326 to the 
multiplexer/demultiplexer 360 where the compressed video data is mixed with the compressed 
audio data for transmission via the channel interface 370 to the ATM network. 

Turning now to Figure 3c, an H.320 codec 1 14c, 134c generally includes a video 
interface 402, 404, an audio interface 406, 408, a data port 410, a slot controller interface 412, 
a video compressor 414, a video decompressor 416, an audio compressor 418, an audio 
decompressor 420, a multiplexer 422 / demultiplexer 424, and a host processor 426. The video 
interface includes an NTSC or PAL video to CCIR-601 format decoder 402 and a CCIR-601 to 
NTSC or PAL encoder 404. The audio interface includes an A/D converter 406 and a D/A 
converter 408. The sampling rates of the converters 406, 408 are preferably programmable 
from 8KHz to 48KHz with either 8 or 16 bit samples. The data port is preferably an RS-232 
interface 410. The slot controller interface 412 includes the same CSP as the Motion JPEG 
codec shown and described with reference to Figure 3a. The interface 412 also includes 
circuitry for formatting/deformatting an H.320 data stream flowing through an ATM adaptation 
processor. The video compressor 414 and decompressor 416 support the CCITT H.261 (full 
CJF and QCIF) recommendation for video codecs. This provides a video frame rate of 15 fps 
with an H.320 serial speed of 384 Kbps. The audio compressor 418 and decompressor 420 
support G.711 (n-law, A-law, PCM), G.722 (ADPCM), and G.728 (CELP) speech codecs 
and are preferably implemented with a DSP. The multiplexer 422 and demultiplexer 424 
support the H.221 portion of the H.320 standard. Video, audio, data, and signalling are 
provided in a signals stream of 64 Kbps with the upper bound being 384 Kbps. The host 
processor 426 controls the H.320 codec and is preferably a Power PC. 
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As mentioned above with reference to Figure 2, each of the codecs in the server 130 can 
be arranged to provide either baseband analog or decompressed digital signals to the audio 
processor and the video processor for mixing and multiplexing according to user commands. 
From the foregoing description of the codecs, it will be appreciated that if the audio and video 
processing is to be performed on decompressed digital signals, the various A/D, D/A, and video 
coder/decoders described in the codecs may be omitted or bypassed. 

There have been described and illustrated herein methods, apparatus, and systems for 
interconnecting compressed multimedia communication data streams having different 
compression formats. While particular embodiments of the invention have been described, it is 
not intended that the invention be limited thereto, as it is intended that the invention be as broad 
in scope as the art will allow and that the specification be read likewise. Thus, while the 
invention has been described with reference to three particular compression standards (Motion 
JPEG, MPEG-2, and H.320), it will be appreciated that it may be appropriate in some 
circumstances to include other standards with appropriate codecs provided that appropriate 
decompressed digital or baseband analog signals can be generated, or to exclude one of the 
disclosed codecs. It will also be appreciated that while the multimedia server of the invention 
has been described with reference to an ATM network and an ISDN network, other network 
architectures could be utilized. It will therefore be appreciated by those skilled in the art that yet 
other modifications could be made to the provided invention without deviating from its spirit 
and scope as so claimed. 
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Claims: 

1. A telecommunications multimedia communications system for use in conjunction with a 
telecommunications network, said system comprising: 

a) a first multimedia terminal coupled to the telecommunications network, said first multimedia 
terminal including a first terminal codec means for generating an outgoing audio/video data 
stream and for receiving an incoming audio/video data stream, said first terminal codec means 
utilizing a first compression standard; 

b) a second multimedia terminal coupled to the telecommunications network, said second 
multimedia terminal including a second terminal codec means for generating an outgoing 
audio/video data stream and for receiving an incoming audio/video data stream, said second 
terminal codec means utilizing a second compression standard; 

c) a multimedia server means coupled to the network for interconnecting said first and second 
terminal codec means, said server means including 

i) a first server codec means for generating an outgoing audio/video data stream and for 
receiving an incoming audio/video data stream, said first server codec means utilizing said first 
compression standard, 

ii) a second server codec means for generating an outgoing audio/video data stream and 
for receiving an incoming audio/video data stream, said second server codec means utilizing 
said second compression standard, 

iii) audio processing means coupled to said first server codec means and to said second 
server codec means for processing audio signals received from said first server codec means 
and said second server codec means and for providing processed audio signals to said first 
server codec means and to said second server codec means, 

iv) video processing means coupled to said first server codec means and to said second 
server codec means for processing video signals received from said first server codec means 
and said second server codec means and for providing processed video signals to said first 
server codec means and to said second server codec means, and 

v) controller means coupled to said first server codec means, said second server codec 
means, said audio processing means, and said video processing means such that 

video signals received from said first server codec means are, after processing 
by said video processing means, directed to said second server codec means for transmission to 
said second terminal codec means, 

video signals received from said second server codec means are, after processing 
by said video processing means, directed to said first server codec means for transmission to 
said first terminal codec means, 
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audio signals received from said first server codec means are, after processing 
by said audio processing means, directed to said second server codec means for transmission to 
said second terminal codec means, and 

audio signals received from said second server codec means are, after processing 
by said audio processing means, directed to said first server codec means for transmission to 
said first terminal codec means. 

2. A telecommunications multimedia communications system according to claim 1, wherein: 

said first and second server codec means provide decompressed digital audio data to said 
audio processing means. 

3. A telecommunications multimedia communications system according to claim 1, wherein: 

said first and second server codec means provide decompressed digital video data to said 
video processing means. 

4. A telecommunications multimedia communications system according to claim 2, wherein: 

said first and second server codec means provide decompressed digital video data to said 
video processing means. 

5. A telecommunications multimedia communications system according to claim 1, wherein: 

said first and second server codec means provide baseband analog audio data to said 
audio processing means. 

6. A telecommunications multimedia communications system according to claim 1, wherein: 

said first and second server codec means provide baseband analog video data to said 
video processing means. 

7. A telecommunications multimedia communications system according to claim 5, wherein: 

said first and second server codec means provide baseband analog video data to said 
video processing means. 

8. A telecommunications multimedia communications system according to claim 1 where the 
telecommunications network is an ATM network, wherein: 

said multimedia server means includes an ATM switch. 
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9. A telecommunications multimedia communications system according to claim 1, wherein: 

said first compression standard is one of Motion JPEG, 
MPEG-2, and H.320, and 

said second compression standard is another of Motion JPEG, MPEG-2, and H.320. 

10. A telecommunications multimedia communications system according to claim 1, further 
comprising: 

d) a third multimedia terminal coupled to the telecommunications network, said third 
multimedia tenninal including a third terminal codec means for generating an outgoing 
audio/video data stream and for receiving an incoming audio/video data stream, said third 
terminal codec means utilizing a third compression standard; wherein 
said multimedia server means further includes 

vi) a third server codec means for generating an outgoing audio/video data stream and 
for receiving an mcoming audio/video data stream, said third server codec means utilizing said 
third compression standard, 

said audio processing means coupled to said third server codec means for processing 
audio signals received from said third server codec means and for providing processed audio 
signals to said third server codec means, 

said video processing means coupled to said third server codec means for processing 
video signals received from said third server codec means and for providing processed video 
signals to said third server codec means, and 

said controller means coupled to said third server codec means, such that 

video signals received from said first and second server codec means are, after 
processing by said video processing means, directed to said third server codec means for 
transmission to said third terminal codec means, 

video signals received from said third server codec means are, after processing 
by said video processing means, directed to said first and second server codec means for 
transmission to said first and second terminal codec means, 

audio signals received from said first and second server codec means are, after 
processing by said audio processing means, directed to said third server codec means for 
transmission to said third terminal codec means, and 

audio signals received from said third server codec means are, after processing 
by said audio processing means, directed to said first and second server codec means for 
transmission to said first and second terminal codec means. 
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1 1. A telecommunications multimedia communications system according to claim 10, wherein: 

said first compression standard is Motion JPEG, 
said second compression standard is MPEG-2, and 
said third compression standard is H.320. 

12. A multipoint multimedia server for use in conjunction with a telecommunications network 
having a first multimedia terminal coupled to the telecommunications network via a first terminal 
codec utilizing a first compression standard and a second multimedia terminal coupled to the 
telecommunications network via a second terminal codec utilizing a second compression 
standard, said server comprising: 

a) a first server codec means for generating an outgoing audio/video data stream and for 
receiving an mcoming audio/video data stream, said first server codec means utilizing the first 
compression standard, 

b) a second server codec means for generating an outgoing audio/video data stream and for 
receiving an incoming audio/video data stream, said second server codec means utilizing the 
second compression standard, 

c) audio processing means coupled to said first server codec means and to said second server 
codec means for processing audio signals received from said first server codec means and said 
second server codec means and for providing processed audio signals to said first server codec 
means and to said second server codec means, 

d) video processing means coupled to said first server codec means and to said second server 
codec means for processing video signals received from said first server codec means and said 
second server codec means and for providing processed video signals to said first server codec 
means and to said second server codec means, and 

e) controller means coupled to said first server codec means, said second server codec means, 
said audio processing means, and said video processing means such that 

video signals received from said first server codec means are, after processing 
by said video processing means, directed to said second server codec means for transmission to 
the second terminal codec means, 

video signals received from said second server codec means are, after processing 
by said video processing means, directed to said first server codec means for transmission to the 
first terminal codec means, 

audio signals received from said first server codec means are, after processing 
by said audio processing means, directed to said second server codec means for transmission to 
the second terminal codec means, and 

audio signals received from said second server codec means are, after processing 
by said audio processing means, directed to said first server codec means for transmission to the 
first terminal codec means. 
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13. A multipoint multimedia server according to claim 12, wherein: 

said first and second server codec means provide decompressed digital audio data to said 
audio processing means. 

14. A multipoint multimedia server according to claim 12, wherein: 

said first and second server codec means provide decompressed digital video data to said 
video processing means. 

15. A multipoint multimedia server according to claim 13, wherein: 

said first and second server codec means provide decompressed digital video data to said 
video processing means. 

16. A multipoint multimedia server according to claim 12, wherein: 

said first and second server codec means provide baseband analog audio data to said 
audio processing means. 

17. A multipoint multimedia server according to claim 12, wherein: 

said first and second server codec means provide baseband analog video data to said 
video processing means. 

18. A multipoint multimedia server according to claim 12 where the telecommunications 
network is an ATM network, said server further comprising: 

f) an ATM switch coupled to said first and second server codec means for coupling said server 
to the network. 

19. A multipoint multimedia server according to claim 12, wherein: 

said first compression standard is one of Motion JPEG, 
MPEG-2, and H.320, and 

said second compression standard is another of Motion JPEG, MPEG-2, and H.320. 
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20. A multipoint multimedia server according to claim 8 where the network includes a third 
multimedia terminal coupled to the network via a third terminal codec utilizing a third 
compression standard, said server further comprising: 

f) a third server codec means for generating an outgoing audio/video data stream and for 
receiving an incoming audio/video data stream, said third server codec means utilizing the third 
compression standard, 

said audio processing means coupled to said third server codec means for processing 
audio signals received from said third server codec means and for providing processed audio 
signals to said third server codec means, 

said video processing means coupled to said third server codec means for processing 
video signals received from said third server codec means and for providing processed video 
signals to said third server codec means, and 

said controller means coupled to said third server codec means, such that 

video signals received from said first and second server codec means are, after 
processing by said video processing means, directed to said third server codec means for 
transmission to the third terminal codec means, 

video signals received from said third server codec means are, after processing 
by said video processing means, directed to said first and second server codec means for 
transmission to the first and second terminal codec means, 

audio signals received from said first and second server codec means are, after 
processing by said audio processing means, directed to said third server codec means for 
transmission to the third terminal codec means, and 

audio signals received from said third server codec means are, after processing 
by said audio processing means, directed to said first and second server codec means for 
transmission to the first and second terminal codec means. 

21. A multipoint multimedia server according to claim 20, wherein: 

said first compression standard is Motion JPEG, 
said second compression standard is MPEG-2, and 
said third compression standard is H.320. 




SUBSTITUTE SHEET (RULE 26) 



WO 99/18728 



PCT/US98/20706 




SUBSTITUTE SHEET (RULE 26) 



WO 99/18728 



PCT/US98/20706 



3/10 



NODE CONTROL 
AND SECURITY 



OA k M 



x 140a 140b ; 
/140c 



AUDIO VIDEO 
CONTROL 



138\ 



SYSTEM 
CONTROL 



DATA PROCESSING 
SYSTEM 



CONFERENCE 
SCHEDULING 



SERIAL 
LINK 



SYSTEM CONTROL BUS 



VIDEO PROCESSING 
SYSTEM 



QUAD 
SPLIT 



138b 7 



/138a 



VIDEO 
SWITCH 



ANALOG 
VIDEO 



AUDIO PROCESSING 
SYSTEM 



DSP 



136b 
136d 



A/D 



1L 



D/A 



ANALOG 
AUDIO 



GCC(T.124) 



137e 



137d 



MCS(T.122) 



T.123 



/137b 



TCP/IP 



/137a 



ETHERNET 
INTERFACE 



I 

ETHERNET 



TO CODECS/ATM SWITCH 



FIG.2a 



SUBSTITUTE SHEET (RULE 26) 



WO 99/18728 



PCT/US98/20706 



GET FROM A/D STAGE THE 
AUDIO SAMPLES FOR All 
PORTS AND STORE THEM 

INTO MATRIX S 
(PORT, SAMPLE NUMBER) 



FOR EACH PORT... 
IF PORT P IS INACTIVE OR 
MUTED, SET ALL ITS 
SAMPLES TO 0 S[P]=0 



FOR EACH GROUP, ADD ALL 

OF ITS MEMBER PORTS 
SAMPLES TOGETHER IN T[gj 



I 



FOR EACH PORT, 
COMPUTE ITS POWER 
FIGURE P[p] 



FOR EACH GROUP, FIND ITS 
LOUDEST PORT L[g] 



FOR EACH PORT, SUBTRACT 
ITS SAMPLE VECTOR FROM 
ITS GROUP VECTOR 
S[P]=T[G]-S[P] 



SEND OUT AUDIO SAMPLES 
FOR ALL POINTS TO THE 
D/A STAGE 



(END) 



4/10 

"1 



.162 



.166 



^168 



FOR EACH PORT... 
IF THE PORT 
IS UNDER AGC, ADJUST GAIN 
ACCORDING TO EFFECTIVE AND 
TARGETTED POWER VALUES 



.170 



.172 



174 




PROCESS COMMAND 
AND SET REQUIRED 
FLAGS FOR ISR 



158 



FIG.2b 



SUBSTITUTE SHEET (RULE 26) 



WO 99/18728 



PCT/US98/20706 



5/10 




SUBSTITUTE SHEET (RULE 26) 



WO 99/18728 



PCT/US98/20706 



T 


<N 




o 


1 

O 


I 

a 










O 


6 




U_ 


Ll 




SUBSTITUTE SHEET (RULE 26) 



WO 99/18728 



PCT/US98/20706 




SUBSTITUTE SHEET (RULE 26) 



WO 99/18728 



PCT/US98/20706 



8/10 



10 

I 

D 



■ LlJ 

^ ^5 £ 



Q_ CD «*J - 

C 

7V 



< 



I c 



SUBSTITUTE SHEET (RULE 26) 



8 



WO 99/18728 



PCT/US98/20706 



9/10 



5 



-Q 











e> 2 




SI 













o 


o 


AUDI 


AUD 



tttt 











lTj 


en 3 




VIDI 
ENCO 


"1 



o 0? £ 



o £5 

5= Si 



ii Ii 



SUBSTITUTE SHEET (RULE 26) 



WO 99/18728 



PCT/US98/20706 



10/10 



3 



II 



o y> uj 
"> H *r 



Of UJ 

II 



£3 



"T 1 f 



SUBSTITUTE SHEET (RULE 26) 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US98/20706 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC(6) :H04N 7/10, 7/14 

USCL :348/12, 13, 441 
According to International Patent Classification (IPC) or to both r. 


ational classification and IPC 




a FIELDS SEARCHED 


Minimum documentation searched (classification system followed by classification symbols) 
U.S. : 348/12, 13. 15. 17. 384, 385. 441; 395/20034. 200.35. 20036. 20037 


Documentation searched other than minimum documentation to tho 


extent that such documents are included 


in the fields searched 


Electronic d 


ata base consulted during the international search (na 


me of data base and. where practicable 


search terms used) 


C. DOCUMENTS CONSIDERED TO BE RELEVANT 


Category* 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


X 


WO 93/16557 Al (KOZ et al) 19 August 1993, p.6, ln.17-31, p.7, 
ln.28, to p.9, ln.7, p. 16, In. 16-37, p. 17, In. 14-30, p. 18, ln.30, to 
p. 19, ln.6, p.24, ln.5-22, p.27, ln.5-25, p.28, ln.4-38, p.30, In. 16- 
38, and p.32, ln.32, to p.33, ln.2. 


1-7, 10, 12-17 


Y 


8, 9, 11, 18-21 


Y 


US 5,555,017 A (LANDANTE et al) 10 September 1996, col.4, 
In. 19-23, andcol.5, ln.23-25. 


8, 9, 18-20 


Y 


US 5,488,433 A (WASHINO et al) 30 January 1996, col.4, ln.64- 
67. 


11, 21 


□ Fur* 


ier documents are listed in the continuation of Box C. Q See patent family annex. 




•A' doou-«t d'f'njni <ti-g.n«r.l tlM. of lb. «1 wh»h u not con.id«.d 

"L* docuamt which mty throw ooubtt on priority ckun(i) or which it 
citod to «ubl»h Ibt publictioo d.u of inoth*r cKiuon or otlwr 
■point rawm (n tpwifnd) 

•P" oocua«ol published prior u> in inuraiuoml fdio« dua but Uur Ihtn 
tr> priority <t»u chinod 


ooiuidarad wjvaToTUnnot bt comidtred to imoWt an in»«rrti»« Map 
■V doowm of psrucuhr whvaiiw; th. thaao^aivwtwnMniiot b» 

SS^bttow'to'I pL^'.kdkd hTa^rt™""* - 00 " 1A " uon 
■a." docuamU moibn or tha una patent family 


Date of the actual completion of the international search 
10 DECEMBER 1998 


Date of mailing of the international search report 

87JAN1999 


Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington. D.C. 20231 
Facsimile No. (703) 305-3230 


Authorized officer «^ . 

BRYAN S. TUNG OdW' "U JLl 
Telephone No. (703) 305-4700 



Form PCT/1SA/210 (second sheetXJuly 1992)* 



